Testing for Neglected Nonlinearity Using Regularized Articial Neural Networks

The arti…cial neural network (ANN) test of Lee, White and Granger (LWG, 1993) uses the ability of the ANN activation functions in the hidden layer to detect neglected functional misspeci…cation. As the estimation of the ANN model is often quite di¢ cult, LWG suggested activate the ANN hidden units based on randomly drawn activation parameters. To be robust to the random activations, a large number of activations is desirable. This leads to a situation for which regularization of the dimensionality is needed by techniques such as principal component analysis (PCA), Lasso, Pretest, partial least squares (PLS), among others. However, some regularization methods can lead to selection bias in testing if the dimensionality reduction is conducted by supervising the relationship between the ANN hidden layer activations of inputs and the output variable. This paper demonstrates that while these supervised regularization methods such as Lasso, Pretest, PLS, may be useful for forecasting, they may not be used for testing because the supervised regularization would create the post-sample inference or post-selection inference (PoSI) problem. Our Monte Carlo simulation shows that the PoSI problem is especially severe with PLS and Pretest while it seems relatively mild or even negligible with Lasso. This paper also demonstrates that the use of unsupervised regularization does not lead to the PoSI problem. LWG (1993) suggested a regularization by principal components, which is a unsupervised regularization. While the supervised regularizations may be useful in forecasting, regularization should not be supervised in inference. Keywords: Randomized ANN activations, Dimension reduction, Supervised regularization, Unsupervised regularization, PCA, Lasso, PLS, Pretest, PoSI problem. JEL Classi…cation: C12, C45 Department of Economics, University of California, Riverside, CA 92521, USA. E-mails: [email protected], [email protected], [email protected].